Empirical Comparisons of Data Flow and Mutation Testing

Data ow and mutation testing are two powerful white box testing techniques for unit-level software testing. Unfortunately, they cannot be completely compared on an analytical basis, for example, mutation is incomparable on an inclusion basis with most data ow criteria. This paper shows that mutation includes the All-defs data ow criterion, but is incomparable with other data ow criteria, and presents results from two empirical comparisons of mutation with the All-uses data ow criterion. For the rst comparison, we de ne a coverage measure that is used for the comparison. The coverage of data ow by mutation is between 99% and 100%, which means that test data that satis ed the mutation criterion also satis ed the data ow criterion in almost all cases. The coverage of mutation by data ow was between 80 and 90%. For the second comparison, we use test data that satisfy the two criteria to detect faults, and compare the criteria on the basis of the faults found. The empirical evidence strongly indicates that while data ow-adequate test sets are close to being mutation-adequate, mutation-adequate test sets are almost invariably data ow-adequate.